Testlet-Based Multidimensional Adaptive Testing
نویسندگان
چکیده
Multidimensional adaptive testing (MAT) is a highly efficient method for the simultaneous measurement of several latent traits. Currently, no psychometrically sound approach is available for the use of MAT in testlet-based tests. Testlets are sets of items sharing a common stimulus such as a graph or a text. They are frequently used in large operational testing programs like TOEFL, PISA, PIRLS, or NAEP. To make MAT accessible for such testing programs, we present a novel combination of MAT with a multidimensional generalization of the random effects testlet model (MAT-MTIRT). MAT-MTIRT compared to non-adaptive testing is examined for several combinations of testlet effect variances (0.0, 0.5, 1.0, and 1.5) and testlet sizes (3, 6, and 9 items) with a simulation study considering three ability dimensions with simple loading structure. MAT-MTIRT outperformed non-adaptive testing regarding the measurement precision of the ability estimates. Further, the measurement precision decreased when testlet effect variances and testlet sizes increased. The suggested combination of the MTIRT model therefore provides a solution to the substantial problems of testlet-based tests while keeping the length of the test within an acceptable range.
منابع مشابه
A Comparison of Item and Testlet Selection Procedures
Testlet response theory (TRT) is a measurement model that can capture local dependency in testlet-based tests. One of the purported advantages of TRT over the more commonly-used polytomous IRT approach to modeling testlet-based tests is that it allows for ad hoc testlet construction in a testlet-based computer adaptive test (CAT). The goal of this study was to investigate the merits of such a C...
متن کاملAdaptive Mastery Testing Using the Rasch Model and Bayesian Sequential Decision Theory
A version of sequential mastery testing is studied in which response behavior is modeled by an item response theory (IRT) model. First, a general theoretical framework is sketched that is based on a combination of Bayesian sequential decision theory and item response theory. A discussion follows on how IRT based sequential mastery testing can be generalized to adaptive item and testlet selectio...
متن کاملThree Multidimensional Models for Testlet-Based Tests: Formal Relations and an Empirical Comparison
متن کامل
Dichotomous or polytomous model? equating of testlet-based tests in light of conditional item pair correlations
The performance of dichotomous and polytomous IRT models in equating testletbased tests was compared in this study. To clarify the conditions under which dichotomous and polytomous item response models produce differing results, the DIMTEST program was used for testing essential unidimensionality, and a bias-corrected index (Final Condcorr) was adapted in this study for measuring local item dep...
متن کاملRobust Automated Test Assembly for Testlet-Based Tests: An Illustration with Analytical Reasoning Items
In many high-stakes testing programs, testlets are used to increase efficiency. Since responses to items belonging to the same testlet not only depend on the latent ability but also on correct reading, understanding, and interpretation of the stimulus, the assumption of local independence does not hold. Testlet response theory (TRT) models have been developed to deal with this dependency. For b...
متن کامل